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Abstract 

Increasing numbers of students taking the SAT have declined to 
identify their race/ethnicity. I examined the impact of non- 
respondents on the validity of reported racial/ethnic differences 
and year-to-year changes in test performance. Using an analysis 
reported by Wainer (1988) and SAT data from 1996 to 2003, I 
confirmed Wainer’s findings that non-respondents prevent 
accurate estimations of group differences based on SAT data. I 
then explored the impact of College Board press release 
information on news reports about the achievement gap. I found 
frequent reports of racial/ethnic differences in SAT scores and 
year-to-year changes in scores but negligible consideration of 
non-respondents. Press releases and media reports should 



include information about non-respondents and their impact on 
accuracy of reported differences based on race/ethnicity. 


Introduction 

The term “achievement gap” has taken on particular and important meanings in the past 
decade. “The achievement gap” has become a shorthand way to refer to differences in 
academic achievement between European Americans and members of minority groups 
who historically have been disenfranchised. For some, the gap refers exclusively to 
differences between African Americans and European Americans. For others, it refers to 
a broader group of students: those who aren’t facile in English, the poor, or members of 
other disadvantaged ethnic groups. 

Regardless of who is included in one’s definition, the literature abounds with descriptions 
of gaps in student performance on test scores, which are probably the most commonly 
used indicators of student achievement (Jencks & Phillips, 1998; Flermstein & Murray, 
1994; Lee, 2000; Kober, 2001). Additional evidence of a gap has been found in data 
related to other indicators of achievement, such as grades, dropout rates, college 
attendance or earnings. (Portes, 1996; Ferguson, 2001; Kane, 1998; Vars & Bowen, 

1 998; Johnson & Neal, 1 998; Roderick & Cambrun, 1 999) National organizations of 
schools dedicated to closing the gap, such as the Minority Student Achievement Network, 
have been formed to address this phenomenon. The benchmark against which minority 
achievement is measured is White/European-American performance, and closing the gap 
usually means increasing minority achievement relative to that of White/European 
Americans. 

Reasons for the gap have been explored; the resulting explanations have been legion. 
Some have focused on the historical legacy of racism, prejudice and segregation (Spring, 
2000; House, 1999). Others have examined factors affecting children’s readiness for 
school and their ability to learn in school, such as the role of poverty and other 
differences associated with SES (Ferguson, 2001 ; Barton, 2003), cultural differences in 
language or in adaptation to school (Mercado, 2001, Portes, 1996; Portes, 1999, Ogbu, 
1999; Ogbu, 2003), family and parenting (Okagaki & Frensch, 1998; McAdoo, 1978). 
Another arena of inquiry has focused on how well education serves its varied 
constituents. Some have looked at inequities in resources and opportunity to learn 
(Kozol, 1991; Barton, 2003; Mickelson, 2001); others have looked at how schools and 
educators respond or relate to student diversity (Spring, 2000; Pollock, 2001 ; Ferguson, 

1 998; Delpit, 1 996; Cohen & Steele, 2002), or how they encourage (or fail to encourage) 
academic excellence (Ogbu, 2003; Barton, 2003). Finally, many have discussed the 
qualities of tests or teachers’ assessments of students that can contribute to extraneous 
differences, i.e. error and bias. (Airasian, 2001, Gould, 1981; Valencia & Suzuki, 2001) 

One common research focus has been change in the gap over time, and change typically 
is examined using test data. Studies have traced the ups and downs of minority student 
test performance, usually compared to that of European Americans. These studies 
employ various types of test scores, most often using NAEP and college admissions 
scores such as the SAT. (Kober, 2001 ; Lee, 2000; Miller, 2003; Powell & Steelman, 

1996) For several decades, NAEP report cards have been issued that describe student 
achievement in eight subject areas; these report cards not only provide a current 



snapshot, but also depict longitudinal change. The accountability movement has triggered 
issuance of school, district and state report cards, and with the passage of the No Child 
Left Behind act, these report cards must include disaggregated data and report 
differences in test performance from one year to the next. Finally, each summer, the 
College Board and ACT issue reports on student performance on college admissions 
tests, notably the SAT and ACT assessment. These reports link student achievement to a 
variety of variables including race/ethnicity. 

NAEP reports are based on the results of samples that were selected to be 
representative of the national population of students. Data from state achievement tests 
are based on the performance of all students at a particular grade level. However, college 
admissions tests are administered on a self-selected set of students who have interest in 
attending college. This self-selection means that the data are not representative of 
students in the nation or a particular state in any year. Furthermore, the percentage of 
high school students who intend to apply to college fluctuates from year to year. One 
consequence is that year-to-year differences in student performance can be based on 
true changes in student knowledge or ability, on changes in who takes the test, or both. 

These sampling issues present serious problems with using college admissions test 
scores to make inferences about the academic qualifications of students in the nation, to 
make comparisons among states, or to track year-to-year differences in test performance. 
ACT and the College Board are careful to warn against using SAT or ACT test results to 
make comparisons among states (College Entrance Examination Board, 2002; ACT, 
2003). ACT’s web site includes information briefs that explain how changes in a school’s 
test performance over time can be influenced by a variety of factors, including who 
decides to take the test (ACT, 2004). The College Board’s guidelines on test use include 
a warning against using aggregate data to make judgments, not only about states, but 
also about schools or districts. However, they are less explicit about comparisons among 
groups of students disaggregated by, among other things, race/ethnicity (College 
Entrance Examination Board, 2002). 

Racial/ethnic comparisons of SAT or ACT data are particularly suspect because each 
year a substantial portion of students do not report their race/ethnicity on the survey 
questionnaires that they complete when registering for the tests they are about to take. 
This is not a recent problem. Fifteen years ago, Howard Wainer (1988) examined the 
impact of this missing information on the accuracy of racial/ethnic comparisons using 
data from SAT’s administered from 1 981 to 1 985. Among his findings are the following: 

• The percentage of students not reporting their race/ethnicity (to be called non- 
respondents here) was substantial each year (12 to 14%)-large enough to be 
called the “second largest minority group” taking the test. 

• Non-respondents, as a group, were not similar to or representative of the students 
who did report their race/ethnicity. 

• While there were variations in the performance of non -respondents from year to 
year and in their performance relative to the nation, their underperformance was 
somewhat consistent (22 to 26 points below the national SAT-V means and 21 to 
28 points below the national SAT-M mean). 

• The error caused by the missing racial/ethnic information for the non-respondents 
overwhelmed any differences detected each year among disaggregated groups as 



well as any changes in test performance gaps over time. 


He concluded, “...the nonresponse to ethnic identifiers is sufficient to introduce noise of a 
greater magnitude than the changes being interpreted as real.” (Wainer, p. 778). 

Wainer’s data suggest that the non-respondent group, while not representative of those 
who did report their race/ethnicity, were nevertheless comparatively stable in terms of 
their numbers and test performance. Since the mid-1990’s, however, this group has 
changed considerably. 

Table 1 reports the numbers of students (in thousands) in each group taking the test each 
year covered by this study. It also reports the combined number of non-white 
respondents. Examination of the table reveals that, with the exception of American 
Indians, the numbers of students in each non-white group increased since 1996. By 
contrast, the numbers of white students increased, then decreased. Finally, both the 
proportion and number of non -respondents more than tripled. 

Table 1. Students (in thousands) from Each Racial/Ethnic Group Taking the SAT by 

Year 


Year 

Am. 

Indian 

Asian 

Am. 

African 

Am. 

Mexicn 

Am. 

Puerto 

Rican 

Other 

Latin/Hisp. 

Other 

Total 

Non- 

White 

White 

No 

Resp. 

1996 

9 

84 

107 

37 

13 

32 

28 

310 

681 

94 

1997 

11 

89 

110 

40 

13 

33 

31 

327 

694 

106 

1998 

10 

94 

115 

41 

14 

36 

36 

345 

704 

123 

1999 

8 

96 

119 

43 

14 

38 

38 

357 

718 

146 

2000 

8 

97 

120 

45 

14 

39 

39 

360 

712 

188 

2001 

8 

102 

121 

47 

14 

40 

39 

370 

704 

202 

2002 

8 

103 

123 

48 

14 

42 

39 

377 

699 

253 

2003 

7 

101 

126 

50 

15 

43 

39 

381 

670 

355 


Figure 1 reports change in the percentage of examinees who were non-respondents on 
the question of racial/ethnic identity. In the mid-nineties, the percentages resembled 
percentages in the data reported by Wainer (1988), but this group has steadily increased 
so that by 2003, they made up one-fourth of all those taking the test. In the process, they 
became, to use Wainer’s parlance, the largest minority group taking the SAT. It should 
be noted that throughout this time period, non-respondents were the only “racial/ethnic” 
group that has been majority male. 
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Figure 1. Percentage of SAT 1 Examinees Not Reporting Their Race/Ethnicity 

In sum, then, the recent escalation of interest in the achievement gap, continued use of 
tests to describe that gap, and the dramatic growth of the non-respondent group raise two 
questions. First, how have students who declined to indicate their race/ethnicity from 
1996 to 2003 resembled those reported by Wainer (1988) and how has this affected the 
validity of his conclusion regarding the amount of noise that overwhelms the use of SAT 
scores as an index of the achievement gap and changes in that gap? Second, are there 
practices in releasing and disseminating information about college admissions test scores 
and racial/ethnic performance on those tests that seem to foster inappropriate use of 
these scores when describing the achievement gap to the public in general and the 
education establishment? 

To answer the first question, I used data released on the College Board’s web site to 
track the test performance of non -respondents and to replicate the study conducted by 
Wainer (1988). To answer the second question, I examined the College Board’s press 
release information from the summer of 2003, each state’s data on non-respondents, 
press releases about the SAT score results for 2003 issued by state departments of 
education throughout the United States, and articles from selected local and national 
newspapers about SAT results. I also revisited articles about the achievement gap that 
have included SAT information. In all cases, I looked for two kinds of information: 1) data 
disaggregated by race/ethnicity and discussion of that data and 2) information about non- 
respondents and their impact on our ability to make inferences about racial/ethic groups 
or differences between groups. 

Question 1 

How have students who declined to indicate their race/ethnicity from 1996 to 2003 
resembled those reported by Wainer (1988), and how has this affected the validity of his 
conclusion regarding the amount of noise that overwhelms the use of SAT scores as an 
index of the achievement gap and changes in that gap? 

Method. Using College-Bound Seniors data reported annually by the College Board on its 
web site, I determined how many students took the test each year and how they 


performed on the test. This permitted a comparison with those examinees employed in 
Wainer’s analysis. Then I employed Wainer’s procedure for estimating the representation 
of white students in the non -respondent group and determining the degree to which these 
non-respondents were affecting the validity of examining test performance disaggregated 
by race/ethnicity. 

For this procedure, Wainer used the numbers of students in each racial/ethnic group and 
the mean SAT-V and SAT-M scores for each group to develop estimates of the 
percentage of non-respondents who were white. He employed this approach to test the 
assumption that the mean scores of the non-respondents from each group were the 
same as those of students who reported their race/ethnicity. Were this to be the case, 
then the reluctance of some to report their race/ethnicity would have little or no effect on 
the validity of differences between groups that were found for those based who did report 
their race/ethnicity. 

For each year, he made two estimates, one based on the math scores and the other 
based on the verbal scores. These estimates were compared for consistency. The 
degree of difference between them indicated how much the assumption that non- 
respondents were like those who responded was violated. It should be noted that this 
estimating procedure yields only rough estimates. 

Wainer estimated the proportion of whites in the non-respondent group using the 
following formula: 


^(^SATwhite) + SATnonwhite) ^SATnonresponse’ 


o 

o 

o 

o 


o 


Where P = the proportion of white non-respondents, 

(1-P) = the proportion of non-white non-respondents, 

M SATwhite = SAT mean self-identified white examinees 

M S ATnonwhite = SAT mean °f examinees identifying themselves as 


members of another group, and 

M sATnnnrp*nnn*P = SAT mean of non-respondent examinees. 


Like Wainer, I used this formula to estimate the percentage of white non -respondents 
twice, once using SAT-Verbal scores and once using SAT-Math scores. I repeated this 
analysis for each year from 1996 to 2003. 


Findings. Table 2 reports the SAT I mean scores for the verbal portion of the test for 
each year from 1 996 to 2003. In general, the amount of change for any self -identified 
group over that time period was negligible. The largest change for self-identifying 
students was an increase in 12 scale -score points in the mean for Asian American 
students. In contrast, the change for non-respondents was dramatic, an increase of 24 
scale-score points. This change in non-respondents’ mean scores also contrasts with the 
means reported by Wainer. The difference between the highest and lowest non- 
respondent means in that study was 10 points, and there was no discernable pattern of 
change. 


Table 2. SAT Verbal Mean Scores for Each Year 



YEAR 

Am. 

Indian 

Asian 

Am. 

African 

Am. 

Mexican 

Am. 

Puerto 

Rican 

Other 

Latin/Hisp. 

Other 

White 

No 

Resp. 

Total 

Non- 

White 

1996 

483 

496 

434 

455 

452 

465 

511 

526 

486 

466 

1997 

475 

496 

434 

451 

454 

466 

512 

526 

489 

466 

1998 

480 

498 

434 

453 

452 

461 

511 

526 

490 

467 

1999 

484 

498 

434 

453 

455 

463 

511 

527 

492 

467 

2000 

482 

499 

434 

453 

456 

461 

508 

528 

495 

467 

2001 

481 

501 

433 

451 

457 

460 

503 

529 

497 

466 

2002 

479 

501 

430 

446 

455 

458 

502 

527 

501 

464 

2003 

480 

508 

431 

448 

456 

457 

501 

529 

510 

466 


Another way to look at non-respondents’ SAT performance is to examine how they 
compared to the national mean. Figure 2 shows that the SAT I verbal performance of 
non-respondents steadily increased so that by 2003, their mean score slightly exceeded 
that of the national mean. In other words, not only has this group grown, but it now 
includes more high-performing examinees. 



HS Graduation Year 


Figure 2. Comparison of SAT Verbal Scores: National Mean vs. Mean of Those Not 

Reporting Their Race/Ethnicity 

Examination of the SAT I results for the mathematics portion of the test yields similar 
results. As can be seen in Table 3, the largest increase in mean scores for any identified 
racial/ethnic group was that for Asian-American scores, an increase of 17 scale-score 
points. By contrast, the mean for non-respondents increased by 31 scale-score points. 
The comparison to the change in the national mean depicted in Figure3 shows an 


increase in scores, with the non-respondents outperforming the national group of 
examinees in 2003. In other words, the non-respondents became an increasingly able 
group with respect to math from 1996 to 2003, a pattern not evident in the math scores 
reported for 1980 to 1985 by Wainer (1988). 

Table 3. SAT Math Mean Scores for Each Year 


YEAR 

Am 

Indian 

Asian 

Am 

African 

Am 

Mexicn 

Am 

Puerto 

Rican 

Other 

Latin/Hisp. 

Other 

White 

No 

Resp. 

Total 

Non- 

White 

1996 

477 

558 

422 

459 

445 

466 

512 

523 

494 

479 

1997 

475 

560 

423 

458 

447 

468 

514 

526 

502 

480 

1998 

483 

562 

426 

460 

447 

466 

514 

528 

503 

483 

1999 

481 

560 

422 

456 

448 

464 

513 

528 

505 

480 

2000 

481 

565 

426 

460 

451 

467 

515 

530 

509 

484 

2001 

479 

566 

426 

458 

451 

465 

512 

531 

510 

484 

2002 

483 

569 

427 

457 

451 

464 

514 

533 

516 

485 

2003 

482 

575 

426 

457 

453 

464 

513 

534 

525 

485 



HS Graduation Year 


Figure 3. Comparison of SAT Math Scores: National Mean vs. Mean of Those Not 

Reporting Their Race/Ethnicity 

These changes in the non-respondents’ mean test performance on the verbal and math 
portions of the SAT I , in addition to the dramatic increase in their numbers, suggest that 
there have been substantial and steady changes in the non-respondent group since the 


mid-1990’s. These changes not only made the group more variable over time, compared 
to their counterparts in the early 1980’s, but also reinforce the need to re-estimate the 
composition of this group with respect to race/ethnicity and compare it to the composition 
of those who do report their race/ethnicity. 

Using the same procedure employed by Wainer (1988), I estimated the percentage of 
whites in the non-respondent group twice for each year covered by this study, once 
based on verbal scores and once based on mathematics scores. Then I used the 
difference between the verbal and mathematics-based estimates to calculate the 
difference in the number of estimated white examinees in the non-respondent group for 
each year. Table 4 reports the results. 

Table 4. Estimates of Percent of White Non-Respondents based on SAT-V and SAT- 
M and Difference in the Estimated Number of White Non-Respondents, based on 
the two estimates 


Year 

Percent based on SAT-V 

Percent based on SAT-M 

Difference in Estimated 
Number of White Non- 
Respondents 

1996 

33.6 

34.3 

643 

1998 

39.5 

47.4 

9,244 

1997 

38.6 

44.7 

6,481 

1999 

41.8 

52.2 

15,062 

2000 

46.3 

54.7 

15,713 

2001 

49.0 

55.3 

12,646 

2002 

58.7 

64.7 

15,105 

2003 

70.0 

81.7 

41,507 


The range of non -respondents who are estimated to be White based on the verbal and 
math means is much more varied for the verbal portion of the test for this time period 
(33.6% to 70.0%) than for the time period reported by Wainer (43% to 51%). For the 
mathematics portion of the test, the estimates based on recent means tend to be larger 
and more varied (34.4% to 81.7%) compared to those reported by Wainer (1988) (29% to 
46%). The differences in the estimated number of white students ranged from very small 
(632) to more than double the largest estimate reported by Wainer (1 988) (41 ,507 vs. 
18,000). 

Wainer develops a hypothetical “extreme case” to illustrate the consequence of these 
differences on the “real” means of African Americans, compared to Whites. By holding 
numbers of examinees and means constant, he estimated that the 1 980 mean verbal 
score for African Americans would have to change about 1 6 points in order for the 
percentage of non-white respondents based on the SAT -V score mean to equal the 
estimated percentage of non -white non-respondents based on the SAT -M score mean. 
Based on 1984 data, the requisite change would have to be 35 points. 









































Following Wainer’s lead, I found that the smallest difference, based on 1996 data, would 
be 1 .8 scale-score points, slightly larger than the 1 -point change reported for 2002 to 
2003. For all other years, the differences are much larger, ranging from 17 points in 1998 
to 122 points in 2003. Hence, Wainer’s conclusion that the error associated with non- 
respondents dwarfed the size of any year-to-year gains reported is confirmed. 

It should be noted, furthermore, that the implications of this finding cannot be applied in a 
consistent fashion to each of the states. The map in Figure 4, developed using SYSTAT’s 
boundary map of the continental United States (SYSTAT, 2000), shows the range of 
percentages of SAT examinees declining to report their race/ethnicity by state. The 
percentages range from a low of 12.4 percent (North Dakota) to a high of 30.4 percent 
(Connecticut), and the percentage of non-respondents is moderately correlated (r =.61) 
with the participation rate reported for each state. Suffice it to say that this analysis would 
need to be carried out for each state in order to determine the impact non-respondents 
have on the racial/ethnic findings in that state. 



Non-response 

■ 35 % 

30 % 

25 % 

20 % 

15 % 

10% 


Figure 4. U.S. Map Showing Percentage of Examinees Declining to Specify their 
Race/Ethnicity on the Student Descriptive Questionnaire Note 1 

Note: Data source: non-respondent data reported by the College Board (2003). 

Question 2. 

Are there practices in releasing and disseminating information about college admissions 
test scores and racial/ethnic performance on those tests that seem to foster inappropriate 
use of these scores when describing the achievement gap to the public in general and 
the education establishment? 

The initial source of information about SAT results for 2003 appears in a College Board 
press release and associated tables, charts and reports that were issued on August 26, 
2003 (College Board, 2003). The press release includes text reporting overall changes in 
performance and participation, differences by gender, changes in performance by 
members of various ethnic groups, upcoming changes in the SAT Verbal test and a 
“snapshot” of test takers. 

Race/ethnicity is included in statements regarding the diversity of the test-taking 
population: 

This year saw the largest increase in the number of SAT takers in more than 
15 years. Thirty-eight percent of SAT takers are first-generation college-bound 


students. The proportion of minority students taking the SAT is at an all-time 
high of 36 percent, up 1 percentage point from last year and 6 points from 10 
years ago. 

“Higher SAT scores, a record number of test-takers, and more diversity add 
up to a brighter picture for American education. While we certainly need to 
make more progress, the fact remains that we are clearly headed in the right 
direction,” said College Board President Gaston Caperton. 

Thirty-six percent of SAT takers in the class of 2003 were minorities. The 
number of Mexican American SAT takers increased by 56 percent between 
1993 and 2003. SAT takers in the Other Hispanic category increased by 50 
percent during the same period. 

Race/ethnicity also figured into statements about test performance: 

The overall verbal scores were aided by a strong showing from Asian 
American SAT takers, whose mean verbal scores were, for the first time, 
higher than the national mean. Additionally, Mexican American and African 
American SAT takers improved their average scores by two points and one 
point, respectively, from a year ago. In fact, virtually all ethnic and racial 
groups showed stronger performance on their verbal scores compared to a 
year ago. 

Accompanying the text of the press release is a set of 18 graphs and tables depicting, 
among other things, the diversity of the test-takers (in one table and one graph), the 
changes in college bound students over time (four graphs/tables), plus racial and ethnic 
differences in high school preparation, grades and test performance (eight graphs/tables). 
Nowhere is there a statement about non-respondents, nor are they included in any table 
or graphic in the press release. This information does appear in Table 4-1 on page 10 of 
the College Bound Seniors report that can be accessed in a box entitled “Archives” next 
to the press release documentation on the web site. 

In other words, while it is possible to find out that many examinees are declining to report 
their race/ethnicity, it is not evident in the materials featured as part of the press release. 
Furthermore, the language of the press release and its featured tables and graphs makes 
assertions about minority and non-minority SAT test takers that may not be true. The 
number of non-respondents simply overwhelms any trends that can be discerned from 
the information about the respondents. 

SAT results are treated as major news by most states’ departments of education, as well 
as the press. On the same day as the College Board’s press release, or soon thereafter, 
29 departments of education in the various states and the District of Columbia issued 
press releases about the SAT results for 2003. (Six issued no press releases in 2003, 
and 16 issued press releases but none about the SAT. Either they reported nothing on 
testing (n=3) or reported about other test results such as NAEP, state tests, and /or ACT 
results (n=1 3).) Of the 29 press releases about the SAT, 1 8 included information about 
race/ethnicity. Sixteen of them included text and/or tables about racial/ethnic test 
performance. Eight of them included comments or tables pertaining to the diversity of 



test -takers. 


The number or percentage of students declining to indicate their race/ethnicity appears in 
five of these press releases and accompanying materials. In three cases, tables include 
non-respondent numbers. The Texas press release reports an increase in the percent of 
non-respondents in Texas and the nation. 

The most detailed set of information appears in the Florida press release and 
accompanying report entitled SAT Trends: Florida and the Nation. The press release 
itself merely comments on the change in scores compared to 2002: 

“Florida’s average verbal score rose two points, due largely to higher scores 
among Hispanic, African-American and Asian males. ’’The report’s summary 
includes several bulleted items about the racial/ethnic composition of the test 
takers. 

Example: “Nationwide, the percentage of minority test takers has also been 
increasing, but at a slower rate than in Florida. In 1988, minorities represented 
23% of the test takers nationwide, about the same as in Florida; by 2003 the 
percentage had increased to 36%, with Asians, whose scores are typically 
well above average, representing 9.6%, compared to 4.4% for Florida.” (p. i) 

SAT Trends introductory summary includes a set of bulleted items entitled “ SAT Scores 
by Racial-Ethnic Groups.” Not all of these bulleted items discuss test performance; they 
also cover diversity of the test-taking group, first language, and income. One item 
provides the most detailed discussion from any press release about non -respondents and 
how they have changed over time: 

An increasing percentage of test takers are declining to indicate their race- 
ethnicity. In 2002, 19% of test takers in both Florida and the U.S. did not do 
so. The number of non-respondents in 2003 rose to 24% for Florida and 25% 
for the U.S. ...In past years, those who did not provide this information had 
lower average scores than those who did. In 2003 the trend was reversed... 

This break in the trend makes any changes in scores by race problematic.... 

“ (P- iv). 

Despite this warning, however, five of 21 pages of tables included in this report focus on 
racial/ethnic differences in one way or another. 

A content analysis of a representative sample of newspapers or televised news reports is 
beyond the scope of this research. However, I did gather articles about the 2003 release 
of SAT scores from newspapers from various cities in the United States; many of them 
have a national presence in the sense that they are distributed nationally, or other 
journalists frequently cite them as sources of news information. 

• Akron Beacon Journal 

• Atlanta -Journal Constitution 

• Boston Globe 

• Chicago Sun -Times 



• Christian Science Monitor 

• Dallas Morning News 

• Denver Post 

• Indianapolis Star 

• Las Vegas Review -Journal 

• Los Angeles Times 

• Miami Herald 

• New York Times 

• Philadelphia Inquirer 

• San Francisco Chronicle 

• Seattle Times 

• St. Petersburg Times 

• Wall Street Journal 

• Washington Post 

Ail 18 of these newspapers published at least one article on the SAT. Twelve of them 
discussed racial/ethnic performance in one or more of the following ways: 

• Differences among groups at the national level 

• Differences among groups at the state or local level 

• Changes in group members’ performance compared to last year or past years 

• The alleged presence of bias in the test (cited FairTest) 

• Possible causes for differences, such as differences in school funding, SES, or 
inequalities in education based on race, ethnicity or income 

Ten of them discussed the participation of students of various racial/ethnic groups in the 
College Board testing program. Some articles mentioned the level of participation; others 
discussed increases in participation. A quotation from the president of the College Board, 
Gaston Caperton, was often included: “Higher SAT scores, a record number of test 
takers, and more diversity add to a brighter picture for American education.” 

News sources designed to serve a national audience, either a lay audience or one of 
educators, were mixed in their reporting on SAT scores for the class of 2003. Major 
national news magazines — Time, Newsweek, and U.S. News and World Report — did not 
report SAT results. However, USA Today included SAT results in its August 27 issue and 
of the 15 paragraphs in the article, five discussed the achievement gap. Education Week, 
in an article dated September 3, 2003, reported on the increase of SAT and ACT 
examinees and the questionable level of college preparation coursework these students 
have had. Part of the article focuses on the level of high school courses these students 
have taken; the last half of the article focuses on racial/ethnic differences in test 
performance and why such differences exist. Black Issues in Higher Education published 
an article on September 1 1 that, like the Education Week article, discusses differences in 
college preparation for students from various racial/ethnic groups. The article includes 
two tables, one based on ACT scores, the other based on SAT scores, that report means 
for disaggregated groups. Of note, the ACT table includes two additional categories: 
"Prefer not to respond" and "No Resp.". The College Board table does not. 

Last, but not least, the United States Department of Education, on August 28, 2003, 
issued a statement about SAT scores as evidence of the achievement gap and disparities 



in the education system. In his October 8, 2003, prepared remarks for the High School 
Leadership Summit, Secretary of Education Rod Paige stated that the disparity continues 
between the education of disadvantaged or low-income students and that of other 
students. The speech goes on to use SAT performance of African American and Hispanic 
students as the sole illustration of this point. 

The only news source that reported on the presence of students who declined to indicate 
their race/ethnicity was The Miami Herald. Its August 29, 2003, article on the SAT scores 
of Broward County School District high school seniors states: 

“Results were broken down by race and ethnicity, but it was difficult to 
determine how Broward was doing in narrowing the achievement gap between 
black and Hispanic and white and Asian students. The reason: Almost 1 in 4 
students chose not to list their race or ethnicity. The number of students who 
didn’t answer increased by almost 100 percent.” 


The source of this insight is unclear. The article refers to Katherine Blasik, Broward’s 
director of research and evaluation. It also reports state results, indicating, perhaps, 
familiarity with Florida’s report on SAT trends. Nevertheless, the readers of the Miami 
Herald were cautioned not to look at the SAT as a source of information about the 
achievement gap. 

Discussion 

Taken together, the College Board’s press release, those of various states, and news 
reported from large urban and national news sources suggest that the information the 
public reads about the SAT is grounded in what the College Board reports in its press 
release about SAT results. If one were to depend on the information provided in the press 
release without referring to the College Bound Seniors tables, several incorrect 
inferences could be made: 

• That the magnitude of “achievement gaps” among racial/ethnic groups can be 
determined from SAT data 

• That changes in the SAT performance of students from various racial/ethnic groups 
can be determined 

• That the diversity of those taking the SAT can be determined and tracked over time 

• That racial/ethnic differences in courses taken, income, first-generation college 
enrollment, high school GPA can all be determined from College Board information 

Wainer’s results make it clear that even when the proportions of non-respondents was 
smaller, there were enough of them to create noise that overwhelms any evidence of 
change over time in test scores reported for students from various racial/ethnic groups or 
any evidence of group differences based on SAT results. The results of this study not 
only confirm Wainer’s results but also show that the noise generated by recent groups of 
non-respondents has increased from overwhelming to deafening. 


While Wainer does not discuss inferences about racial/ethnic differences of other kinds, 



the magnitude of the non-respondent rate and the change in this group’s test 
performance suggest that the other inferences associated with racial/ethnic differences 
reported in the College Board’s press release should be treated as suspect as well. The 
graphs in Figure 4 illustrate why this should be the case. The first two breakdowns report 
take-takers based on data reported in the College Board’s press release and College 
Bound Seniors for the class of 2003; the third and fourth breakdowns make use of the 
estimates of white non-respondents from Table 4. 

The lower estimate, based on SAT -V 2003 score means, yields a percentage of white 
test-takers in Breakdown 3 that exceeds by one percent that reported in the College 
Board’s press release with non-respondents removed. Coincidentally, the “increased 
diversity” from 2002 to 2003 was also one percent. Note that the remaining percentage of 
non-respondents (those who are estimated to be non-white) is larger than any of the 
minority groups, except African Americans. 

The estimate based on math score means yields a percentage of white test-takers in 
Breakdown 4 that more substantially exceeds that reported in the press release. Indeed it 
all but negates the increase in minority representation among test-takers reported in the 
press release (from 30% minority in 1993 to 36% in 2003). The proportion of estimated 
minority non-respondents is larger than any of the groups of minority respondents except 
African Americans and Asian Americans. 


Basis for Breakdown of Racial/Ethic 
Groups 

Breakdown 1 . Based on breakdown 
reported in the College Board press 
release 


Pie Chart Associated with Each Basis 
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Breakdown 2 Based on data as 
reported in College Bound Seniors 
with actual non-respondents included 



Breakdown 3. Estimated breakdown 
based on SAT-V analysis 
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Breakdown 4. Estimated breakdown 
based on SAT-M analysis 
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Figure 5. Breakdowns of the Race/Ethnicity of SAT-1 Test Takers from the Class of 
2003 Based on Different Estimates Note 2 

These results suggest that non -respondents prevent valid use of data from College 
Bound Seniors to describe majority and minority test-takers with respect to: 

• The size of the achievement gap 





• Changes in the magnitude of the achievement gap 

• The diversity of those taking the SAT 

• Differences in GPA among racial/ethnic groups 

• First-generation college attendance 

• High school preparation 

Recommendations 

The combined results from this study suggest several further steps. 

First, we need to find out more about these non-respondents. Who are they? Why are 
they declining to indicate their race/ethnicity? Several hypotheses have surfaced. 

One idea, derived from the growth in resistance to affirmative action, is that the non- 
respondents are white males who see no benefit in reporting their race/ethnicity. It is 
documented that non-respondents are majority male and both sets of estimates 
appearing in Table 4 suggest a dramatic increase in white non-respondents. 

A second suggestion is that students from various groups are resisting reporting their 
race/ethnicity as a protest against the use of a category they regard as arbitrary and 
grounded in mistaken notions about race. 

A third suggestion is based on findings of Claude Steele and others pertaining to 
stereotype threat. (Aronson, 2002; Steele & Aronson, 1998) The notion that reporting 
one’s race during a test affects test performance may be filtering its way into the middle- 
class, professional African-American community. Hence, increasing numbers of high- 
achieving minority students may not be revealing their race on test applications. 

In all probability, there are multiple reasons, as might be inferred from a brief analysis of 
the 2003 data from my district. It was possible to identify the race/ethnicity of all students, 
match that to their SAT scores and then compare this racial/ethnic and test score data to 
that from the College Board’s College Bound Seniors 2003 report for Shaker Heights 
High School The data from the College Bound Seniors report for the high school reveal a 
23 percent non-response rate in 2003, which is similar to our state and the national data 
in terms of percentage. Our non-respondents, unlike their national and state counterparts, 
are majority female. Compared to those reporting their race/ethnicity, a disproportionate 
number of our non-respondents are White/European -American. Eighty-two percent of 
them are White; 18 percent are Black/African- American. By contrast, 36 percent of our 
high school’s SAT test takers in the class of 2003 are Black/African-American; 59 percent 
of them are White/European-American. The non-respondent verbal mean was higher 
than the overall mean, and both groups of non-respondents had higher SAT-V scores 
than their counterparts. The math scores, on the other hand, present a contradiction. 

While the White non-respondents had higher SAT-M scores, their Black/African-American 
counterparts had lower SAT-M scores. 

Second, while descriptions of racial/ethnic differences and score changes over time have 
always been questionable, the sheer size of the non-respondent group makes use of SAT 
data for such purposes irresponsible. At the very least, test sponsors and state 
departments of education need to be clear in their press releases about the presence of 



non-respondents and their impact on one’s ability to make inferences about students 
taking college admissions tests in the same way they warn about other misuses of test 
results. The College Board and ACT provide very clear cautions against using their data 
to make inferences about state rankings, for example, and these cautions seems to have 
taken hold. Education departments’ press releases and newspaper articles often make 
adjustments that seem to heed this caution. For example, they reported data only for 
other states with similar participation rates, or they warn readers that participation rate 
and mean scores are strongly related to each other. Furthermore, test companies’ press 
releases themselves need to provide clear information about non-respondents — the 
number of them, their known characteristics compared to respondents (i.e. test 
performance, gender). 

Finally, those who have used these data in news articles, policy papers, or published 
research need to reconsider their use. The difference in the opportunities and 
achievement of children from various racial/ethnic groups is perhaps the most important 
issue facing education today. We need to examine and describe these differences, but 
we need to do it with data that we can count on to provide an accurate picture. Alternative 
data sources such as state data and NAEP results can do the job much more effectively. 

References 


Aronson, J. (2002) Stereotype threat: Contenting and coping with unnerving expectations. In J. Aronson (Ed.), 
Improving academic achievement (pp. 279-301 ). San Diego CA: Academic Press. 

ACT. (2004) How significant are changes in my school’s average ACT composite scores over time?, ACT 
Research information brief 98-3. Retrieved January 9, 2004 from http://www.act.org/research/briefs/98- 
3.html. 

ACT. (2004) Monitoring changes in high school ACT composite scores over time, ACT Research information 
brief 2000-2. Retrieved January 9, 2004 from http://www.act.org/research/briefs/2000-2.html. 

ACT. (2003) ACT national and state scores. Retrieved January 9, 2004 from 
http://www.act.org/news/data/03/index.html 

Airasian, P. (2001) Classroom assessment, 4th ed. Boston: McGraw Hill. 

Barton, P.E. (2003) Parsing the achievement gap (ETS Policy Information Center Report). Princeton, NJ: 
Educational Testing Service 

Cohen, G.L. & Steele, C.M. (2002), A barrier of mistrust: How negative stereotypes affect cross -race mentoring. 
In J. Aronson (Ed.), Improving academic achievement (pp. 303-327). San Diego CA: Academic Press. 

College Board. (2003) SAT verbal and math scores up significantly as a record-breaking number of students 
take the test. Retrieved January 9, 2004 from 
http://www.coilegeboard.eom/press/articie/0, 31 83,26858,00.html 

College Entrance Examination Board. (2002) Guidelines on the used of college board test scores and related 
data. New York: College Entrance Examination Board. 

Ferguson, R.F. (2001 ) A diagnostic analysis of black-white GPA disparities in Shaker Heights, Ohio. In D. 
Ravitch, (Ed.), Brookings Papers of Education Policy, 2001 (pp. 417-414). Washington, DC: Brookings 
Institution Press. 


Ferguson, R.F. (1998) Teachers’ perceptions and expectations and the black -white test score gap. In C. Jencks 



& M. Phillips, (Eds.), The black-white test score gap (pp. 318-374). Washington, DC: Brookings Institution Press 


Gould, S.J. (1981) The mismeasure of man. New York: W.W. Norton & Company. 

Herrnstein, R. J. & Murray, C. (1994) The bell curve: Intelligence and class structure in American life. New York: 
Free Press. 

House, E.R. (1999) Race and policy. Education Policy Analysis Archives, 7 (16). Retrieved August 11, 2000, 
from http://epaa.asu.edu/epaa/v7n1 6.html . 

Jencks, C & Phillips, M. (1998) The black-white test score gap: An introduction. In C. Jencks & M. Phillips, 

(Eds.), The black-white test score gap (pp. 1-51). Washington, DC: Brookings Institution Press 

Johnson, W.R. & Neal, D. (1998) Basic skills and the black-white earnings gap. In C. Jencks & M. Phillips, 

(Eds.), The black-white test score gap (pp. 480-497J. Washington, DC: Brookings Institution Press 

Kane, T.J. (1998) Racial and ethnic preferences in college admissions. In C. Jencks & M. Phillips, (Eds.), The 
black-white test score gap (pp. 431 -456). Washington, DC: Brookings Institution Press. 

Kober, N. (2001 ) It takes more than testing. A report of the Center on Education Policy. Washington, DC: Center 
on Education Policy. 

Lee, J. (2002). Racial and ethnic achievement gap trends: Reversing the progress toward equity? Educational 
Researcher , 31 (1), 3-12. 

McAdoo, H.P. (1978) Factors related to stability in upwardly mobile black families. Journal of Marriage and the 
Family. 40 (November) 761 -776. 

Mercado, C. I. (2001) The learner: “Race”, “ethnicity,” and linguistic differences. In V. Richardson, (Ed.), 
Handbook of research on teaching (pp. 668-694). Washington, DC: American Educational Research 
Association. 

Mickelson, R.A. (2001) Subverting Swann: First- and second-generation segregation in the Charlotte- 
Mecklenburg schools. American Educational Research Journal, 38 (1 ) 215-252. 

Miller, G.E. (2003) Analyzing the minority gap in achievement scores: Issues for states and federal government. 
Educational Measurement: Issues and Practice, 22 (3), 30-36. 

Ogbu, J.U. (1999) Beyond language: Ebonics, proper English, and identify in a Black-American speech 
community. American Educational Research Journal , 36 (2), 147-184. 

Ogbu, J.U. (2003) Black American students in an affluence suburb, A study of academic disengagement. 
Mahwah, NJ: Lawrence Erlbaum Associates. 

Okagaki, L. & Frensch, P.A. (1998) Parenting and children’s school achievement: A multiethnic perspective. 
American Educational Research Journal, 35 (1), 124-144. 

Pollock, M. (2001 ) How the question we ask most about race in education of the very question we most 
suppress. Educational Researcher, 30 (9), 2-12. 

Portes, P. (1 996) Ethnicity and culture in educational psychology. In Berliner & Calfee, eds. Handbook of 
educational psychology (pp. 331 -357). New York: Simon & Schuster Macmillan. 

Portes, P. (1999) Social ansd psychological factors in the academic achievement of children of immigrants: A 
cultural history puzzle. American Educational Research Journal, 36 (3), 489-507. 


Powell, B. & Steelman, L.C. (1996) Bewitched, bothered, and bewildering: The use and misuse of state SAT 
and ACT scores. Harvard Educational Review, 66 (1), 27-59. 

Roderick, M. & Cambrun, E. (1999) Risk and recovery from course failure in the early years of high school. 
American Educational Research Journal. 36 (2), 303-343. 

Spring, J. (2000,). American education, 9 th ed. Boston: McGraw Hill. 

Steele, C.M. & Aronson, J. Stereotype threat and the test performance of academically successful African 
Americans. In C. Jencks & M. Phillips, (Eds.), The black-white test score gap (pp. 410-427). Washington, 
DC: Brookings Institution Press. 

SYSTAT (2000). SYSTAT 10 graphics. Chicago: 2000. 

Valencia, R.R. & Suzuki, LA. (2001) Intelligence testing and minority students . Thousand Oaks, CA: Sage 
Publications. 


Vars, F. E. & Bowen, W.G. (1998) Scholastic Aptitude Test scores, race, and academic performance in 

selective colleges and universities. In C. Jencks & M. Phillips, (Eds.) The black-white test score gap (pp. 
457-479). Washington, DC: Brookings Institution Press 

Wainer, H. (1988) How accurately can we assess changes in minority performance on the SAT? American 
Psychologist, 43 (1 0), 774-778. 


Notes 

1 . Percentages of non -respondents from Alaska, Hawaii and the District of Columbia 
were 26.6%, 20.2% and 28.5% respectively. 

2. Based on aggregated data reported by the U.S. Census Bureau for the 2000 
census, whites made up 63% of 18-year olds in 2000. The second largest group 
was Hispanic (16.1%), followed by African Americans (14.1%). This age group was 
majority male (51 .3%) in 2000. 
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